Explatory Data Analysis for 2ForcedChoice Modal Experiments

1 Procedure for both experiments

• Platform and environment
  • The experiments were conducted online in PCIbex (PennController for Ibex). The interface switched to fullscreen at the start of the main task and exited fullscreen at the end.
  • Participants were instructed to use a desktop or laptop with Google Chrome, a keyboard, and a mouse or trackpad in a distraction-free setting.
  • Total session time was approximately 25 minutes.
• Consent and demographics
  • After an introduction screen, participants viewed and downloaded a consent form and indicated consent to proceed.
  • A brief demographics form collected age, gender, location (state and country), computer type, native language, and other languages.
• Design and counterbalancing
  • Aim: the experiments tested the interpretive force of must compared to other modal elements, or the absence of a modal.
  • Participants were randomly assigned (between subjects) to one of four answer sets: bare, have to, will, prob (a separate CSV file per set).
  • Experimental, filler, and attention-check trials were fully intermixed and randomized for each participant.
  • On critical trials, the modal from a participant’s assigned set was contrasted with must (for example, will vs. must).
  • Filler trials featured the remaining modal options among the two responses.
  • Attention-check trials consisted of unambiguous deductive contexts to verify attention.
  • Counts: experimental items = X; fillers = X; attention checks = X.
  • Exclusion criterion: participants were excluded if they failed more than 20% of attention-check trials.
  • Each experimental item appeared in one of three context conditions: abductive, deductive, or inference.
  • Context conditions were assigned using a Latin square, so each participant saw exactly one context per item.
  • On every trial, the two response options were order-randomized.
  • Between-experiment differences:
    • Experiment 1 used SONA participants and included preambles that conveyed contextual information in an if-clause.
    • Experiment 2 used Prolific participants and included a shorter preamble consisting only of a well, then… phrase.
• Instructions and practice
  • Participants read step-by-step instructions with two worked examples (one obvious and one less obvious).
  • Additional practice trials followed, using the same two-stage flow as the main task.
• Trial structure (main task)
  • Context presentation: participants saw a short, three-turn conversation (Speaker A – Speaker B – Speaker A) that provided the relevant context.
  • Dialogue (reading) phase: participants pressed the space bar to advance or were auto-advanced after 120 s. A Dialogue RT (ms) was recorded from dialogue onset to advance.
  • Choice phase: two candidate sentence completions appeared simultaneously. Participants selected the option they judged most appropriate; a 60 s timeout applied. An Answer RT (ms) was recorded from option onset to response.
  • After the response, the screen cleared and the next trial began.
• Breaks
  • Brief on-screen breaks were inserted at regular intervals (about every four trials). Participants resumed by pressing the space bar.
• Measures and logging
  • For each trial, the following were logged: item and condition labels (Type, Group, Item, Inference, ConditionName, Condition), the three dialogue lines (D1–D3), both options (A1–A2), the selected answer, Dialogue RT, Answer RT, and TrialNumber.
  • Practice trials were logged with the same structure and marked as practice.
• Debrief and redirect
  • Upon completion, results were submitted to the server, fullscreen ended, and participants were automatically redirected to a debrief and credit form hosted at UMD.

2 Experiment 1

2.1 Participant Accuracy in Check Items

must vs. prob
Total N = 14
Bin	Count
[0.5,0.6)	3
[0.7,0.8)	6
[0.9,1]	5

must vs. will
Total N = 6
Bin	Count
[0.7,0.8)	1
[0.9,1]	5

haveto vs. must
Total N = 9
Bin	Count
[0.7,0.8)	2
[0.9,1]	7

bare vs. must
Total N = 11
Bin	Count
[0.9,1]	11

Click to expand Demographics

must vs. prob

Mean Age: 22.00 (18–49)

Subject	Age	Gender	Location	Computer	Language	Other language
S[51]	18	Male	Maryland, United States of America	Mac	English	Spanish (proficient), Darija (novice)
S[55]	30	Male	MD, USA	Mac	English
S[60]	18	Unlabeled	Maryland, United States	PC (Lenovo)	English	Cantonese
S[5]	19	Female	Maryland, USA	PC	English	Hindi, Urdu
S[6]	18	Female	MD, USA	Windows	English
S[9]	21	Female	Maryland, USA	Mac	English
S[12]	49	MALE	MARYLAND, USA	LINUX PC	ENGLISH	SPANISH, ITALIAN, RUSSIAN NONE FLUENTLY
S[17]	19	Female	Maryland, United States	Windows Laptop	English	Spanish
S[21]	18	Male	Maryland	Laptop	English
S[22]	18	Male	Maryland, United States	PC	English
S[26]	18	Female	Maryland, USA	Lenovo	English
S[27]	20	Female	Maryland, USA	Windows	English	Mandarin Chinese
S[31]	18	Female	College Park, Maryland	Mac	English
S[34]	24	Female	Maryland, United States	Mac	English	Korean

must vs. will

Mean Age: 21.67 (18–36)

Subject	Age	Gender	Location	Computer	Language	Other language
S[53]	18	male	maryland, usa	pc	english
S[57]	18	female	Maryland, USA	laptop	english
S[59]	18	Female	Maryland, USA	Mac	English	Spanish
S[10]	21	Male	Maryland, Prince George’s County	PC	English	Spanish
S[11]	19	Male	Maryland, US	Windows	English
S[20]	36	Male	Baltimore, MD	Mac	English	French, German

haveto vs. must

Mean Age: 19.00 (18–22)

Subject	Age	Gender	Location	Computer	Language	Other language
S[56]	18	female	Maryland, USA	lenovo thinkpad	english	chinese (beginner)
S[58]	19	Female	MD, USA	Mac	English
S[1]	18	Female	College Park, Maryland	Mac	English	Cantonese
S[13]	19	Male	College Park, MD, USA	Mac	English	Gujarati
S[15]	20	Male	College Park, MD	Mac	English	Telugu
S[16]	22	Male	MD, USA	PC	English	German
S[28]	18	Female	Maryland, USA	Mac	English	Spanish
S[30]	19	Male	Maryland, United States of America	PC	English
S[35]	18	Female	College Park, MD	Mac	English	Mandarin Chinese

bare vs. must

Mean Age: 21.27 (18–40)

Subject	Age	Gender	Location	Computer	Language	Other language
S[52]	18	male	MD, USA	Thinkpad X1 Carbon	English	Mandarin, Spanish, Japanese, Hindi
S[54]	20	Female	Maryland, United States	PC	English
S[3]	20	Male	Massachusetts	PC	English
S[7]	18	woman	MD, USA	Windows	English	Japanese
S[8]	40	Female	Maryland, USA	PC	English
S[14]	19	Female	Maryland, United States	Mac	English	Vietnamese
S[18]	26	male	Maryland, USA	PC	English	Spanish (L2 ~B2/C1)
S[19]	19	Female	Maryland, USA	PC	English	Spanish
S[24]	18	Female	Maryland, United States	Mac	English
S[25]	18	female	Maryland, USA	Dell laptop	English	Vietnamese
S[32]	18	female	MD, United States	Mac	English

2.2 Answer Choice Summary

2.3 Reading Times Summary

3 Experiment 2

3.1 Participant Accuracy in Check Items

must vs. prob
Total N = 21
Bin	Count
[0.5,0.6)	1
[0.9,1]	20

must vs. will
Total N = 21
Bin	Count
[0.7,0.8)	1
[0.9,1]	20

haveto vs. must
Total N = 21
Bin	Count
[0.9,1]	21

bare vs. must
Total N = 21
Bin	Count
[0.7,0.8)	3
[0.9,1]	18

Click to expand Demographics

must vs. prob

Mean Age: 45.86 (27–73)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	44	male	TN, USA	HP PC	English
S[2]	49	male	Michigan, USA	PC	English
S[3]	46	male	texas, USA	PC	English
S[4]	52	male	Tennessee, USA	Mac	English
S[5]	32	Male	NC, USA	PC	English
S[6]	59	Female	Alabama	P.C.	English
S[7]	73	Female	Rhode Island	PC	English
S[8]	47	Male	NY, USA	PC	English
S[9]	41	male	MA, USA	PC	English
S[10]	48	Woman	NY, USA	PC	English
S[11]	56	male	Virginia, USA	PC	English
S[12]	50	female	California, United States	laptop	English
S[13]	64	Male	Illinois, United States	PC	English
S[14]	30	Female	Hawaii, USA	PC	English
S[15]	34	Female	New York, United States of America	PC	English
S[16]	40	Female	New York, USA	Laptop	English
S[17]	42	Male	Kentucky, USA	Windows laptop	English
S[18]	37	male	Indiana, United states	PC	English
S[19]	27	woman	US, TEXAS	WINDOWS	ENGLISH
S[20]	32	man	Tennessee, United States	PC	English	Spanish
S[21]	60	Male	North Carolina, United States	PC	English

must vs. will

Mean Age: 46.10 (26–67)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	31	female	NJ USA	Mac	English
S[2]	52	Female	Minnesota, USA	PC	English	Beginning Spanish
S[3]	54	Male	CA, USA	PC	English
S[4]	30	Female	Kentucky, USA	PC	English
S[5]	43	female	Connecticut, USA	PC	English
S[6]	26	male	north carolina, united states	pc	english
S[7]	55	female	Tennessee, USA	Windows Laptop	English
S[8]	37	Female	Georgia, United States	PC	English
S[9]	36	female	Maryland United states	Chromebook	English
S[10]	43	Female	Texas, USA	Mac	English
S[11]	67	female	Wisconsin, USA	PC	English	Spanish
S[12]	46	male	tn, usa	PC	English
S[13]	49	Female	Massachusetts	PC	English
S[14]	56	female	california united states	pc	english
S[15]	62	male	ny, usa	pc	English
S[16]	51	Female	Virginia	Laptop	English
S[17]	28	Female	Maine, USA	PC	English
S[18]	59	male	Tennessee, USA	PC	English
S[19]	52	female	Virginia	PC	English	English
S[20]	29	Female	California United States	Laptop	English
S[21]	62	female	Washington, USA	PC	English

haveto vs. must

Mean Age: 37.24 (22–71)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	24	Male	Florida, USA	PC	English
S[2]	45	Male	Oregon, USA	PC	English
S[3]	29	f	fl, usa	pc	english
S[4]	22	F	CO, USA	PC	ENGLISH
S[5]	32	male	Minnesota, USA	PC	English
S[6]	27	female	Georgia, USA	PC	English
S[7]	38	Female	Louisiana, United States	PC	English
S[8]	33	Woman	Maryland, USA	Mac	English
S[9]	58	Male	Georgia, USA	PC	English
S[10]	45	Male	MS, USA	MAC	English
S[11]	53	Female	United States	Mac	English	English
S[12]	35	Non-Binary AFAB	MN, USA	Mac	English
S[13]	42	woman	Tennessee, USA	PC(windows)	English
S[14]	30	Female	Indiana, USA	PC	English
S[15]	41	Female	California, USA	Mac	English
S[16]	28	Male	PA, USA	Linux	English
S[17]	47	Male	Michigan, United States	PC	English
S[18]	71	Female	Arizona, United States	PC	English
S[19]	26	Female	Illinois, USA	PC	English Language
S[20]	25	Woman	Delaware	PC	English
S[21]	31	Male	MA, USA	Windows	English

bare vs. must

Mean Age: 43.00 (21–66)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	36	male	CA, USA	PC	English
S[2]	35	Male	AL, Geneva	Chromebook	English	No others
S[3]	38	male	Michigan, United States Of America	PC	English
S[4]	55	male	California, USA	PC	English
S[5]	51	Male	New York, United States	PC	English
S[6]	61	female	California, US	PC	English
S[7]	21	male	Georgia, United States	PC	English
S[8]	35	male	VA, USA	windows PC	English
S[9]	45	Male	Ohio, United States	PC	English
S[10]	66	Female	Florida, United States	PC	English
S[11]	44	Male	GA, USA	PC	English
S[12]	40	Female	Louisiana, United States	PC	English
S[13]	25	Male	United States	PC	English
S[14]	47	male	New York, United States	PC	English
S[15]	46	Female	Nevada, USA	PC	English
S[16]	47	Female	Mississippi, United States	PC	English
S[17]	43	Woman	SC, United States	Windows	English
S[18]	29	female	ohio, USA	PC	english
S[19]	39	Male	United States	PC	English
S[20]	45	Male	Indiana, USA	Laptop	English
S[21]	55	female	NY	PC	English	English

3.2 Answer Choice Summary

3.3 Reading Times Summary

3.4 Age groups

4 Experiment 3

4.1 Participant Accuracy in Check Items

haveto vs. must
Total N = 21
Bin	Count
[0.9,1]	21

bare vs. must
Total N = 21
Bin	Count
[0.7,0.8)	1
[0.9,1]	20

Click to expand Demographics

haveto vs. must

Mean Age: 48.57 (35–73)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	37	male	California, USA	PC	english
S[2]	60	Female	Idaho, USA	PC	English
S[3]	47	MALE	BURKETVILLE, MD	PC	ENGLISH	NONE
S[4]	38	male	ma, usa	mac	english
S[5]	60	Female	CA, USA	Mac	English
S[6]	50	Female	California, USA	Mac	English
S[7]	42	female	RI, USA	PC	English
S[8]	73	Female	Kansas, USA	laptop	English
S[9]	35	male	Colorado, United States	PC	English
S[10]	49	Male	MN	Mac	English
S[11]	46	male	Washington, USA	pc	English
S[12]	42	Female	Virginia, USA	PC	English
S[13]	72	female	Ohio, United States	PC	English
S[14]	44	female	IL, USA	PC	English
S[15]	46	Male	Michigan, United States	PC	English
S[16]	52	male	CT USA	pc	English
S[17]	41	male	Indiana. United states	Laptop	English
S[18]	58	female	Michigan, United States	PC	English
S[19]	39	male	NH, USA	PC	English
S[20]	44	female	California, US	1pc	English
S[21]	45	Male	NC, USA	PC	English	French (Intermediate), Spanish (Basic)

bare vs. must

Mean Age: 48.52 (36–72)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	49	woman	NY, USA	PC	English
S[2]	43	Female	New York, US	HP	English
S[3]	38	Female	New York, United States	PC	English
S[4]	38	Male	Arlington, USA	PC	English	English
S[5]	71	Male	Texas, United States	PC	English
S[6]	43	F	MA, USA	Mac	English
S[7]	45	Female	Maryland, USA	Laptop%2C Windows	English
S[8]	37	female	La Porte, Texas	PC	English
S[9]	47	Male	Florida, USA	PC	English
S[10]	52	female	Missouri, USA	Mac	English
S[11]	51	male	New Mexico, USA	PC	English
S[12]	44	Female	NJ, United States	PC	English
S[13]	51	Male	Nevada, USA	PC	English	0
S[14]	48	male	OH, USA	PC	English
S[15]	36	female	California, usa	pc	english
S[16]	72	Female	Louisiana, United States	PC	English
S[17]	57	woman	CA USA	Mac	English
S[18]	51	female	michigan	pc	english
S[19]	60	male	Missouri, USA	PC	American English	Patios
S[20]	39	male	RI, USA	Pc	English
S[21]	47	Male	PA, USA	MAC	English

4.2 Answer Choice Summary

4.3 Reading Times Summary

5 Experiment 4

5.1 Participant Accuracy in Check Items

gotta vs. must
Total N = 21
Bin	Count
[0.7,0.8)	2
[0.9,1]	19

gotta vs. must
Total N = 21
Bin	Count
[0.7,0.8)	2
[0.9,1]	19

Click to expand Demographics

gotta vs. must

Mean Age: 46.00 (35–65)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	65	female	Arkansas, USA	Windows	English
S[2]	42	female	MO, USA	Mac	English
S[3]	51	Female	California, USA	PC	English
S[4]	62	male	florida, usa	pc	english
S[5]	43	Female	Kansas, USA	PC	English
S[6]	51	female	IL, United States	PC	English
S[7]	35	Female	OH, United States	PC	English
S[8]	50	male	Texas, USA	pc	English	Spanish
S[9]	39	female	Willis, TX	PC	English
S[10]	39	Male	Wyoming, USA	PC	English
S[11]	50	male	New Jersey, USA	PC	English
S[12]	38	female	Oklahoma, USA	PC	English
S[13]	37	Male	IL, USA	PC	English
S[14]	37	Male	New Jersey; United States	Desktop PC	English
S[15]	54	Female	Illinois, United States	Laptop	English
S[16]	48	Male	NH	PC (windows)	English
S[17]	40	Male	Maryland, United States	PC	English
S[18]	43	Male	Illinois, USA	PC	English
S[19]	50	male	California, USA	pc	English	Spanish
S[20]	37	Female	North Carolina, USA	PC	English
S[21]	55	Female	New York	PC	English

gotta vs. must

Mean Age: 50.05 (35–81)

Subject	Age	Gender	Location	Computer	Language	Other language
S[1]	41	female	PA, USA	PC	English
S[2]	38	Male	Illinois, USA	PC	English
S[3]	48	Male	Wisconsin, USA	PC	English
S[4]	59	woman	US	PC	English
S[5]	62	Female	California, USA	PC	English
S[6]	43	male	Florida, USA	PC	English
S[7]	47	Male	Georgia, United States	PC	English
S[8]	51	male	NC, Bladen	PC	English
S[9]	81	Female	Tennessee, USA	laptop	English
S[10]	43	female	Michigan, United States	PC	English
S[11]	44	Male	Michigan, United States	PC	English
S[12]	38	male	NC, USA	PC	English
S[13]	39	Male	Kansas, United States	PC	English
S[14]	55	female	Florida, USA	PC	English
S[15]	53	Woman	Ohio, United States	Mac	English
S[16]	49	male	florida, USA	PC	English
S[17]	53	Woman	GA, United States	PC Windows	English
S[18]	47	Male	Indiana, USA	PC	English
S[19]	65	female	North Carolina, USA	PC	English
S[20]	60	Female	MA, USA	PC	English
S[21]	35	female	CA, US	Mac	English

5.2 Answer Choice Summary

5.3 Reading Times Summary